In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
#Reading the training images from the path and labelling them into the given categories
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cv2 # this is an important module to get imported which may even cause issues while reading the data if not used
import seaborn as sns # for data visualization
import tensorflow as tf
import keras
import os
# Library to split data
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.models import Sequential #sequential api for sequential model
from tensorflow.keras.layers import Dense, Dropout, Flatten #importing different layers
from tensorflow.keras.layers import Conv2D, MaxPooling2D, BatchNormalization, Activation, Input, LeakyReLU,Activation
from tensorflow.keras import backend
from tensorflow.keras.utils import to_categorical #to perform one-hot encoding
from tensorflow.keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D
from tensorflow.keras.optimizers import RMSprop,Adam,SGD #optimiers for optimizing the model
from tensorflow.keras.callbacks import EarlyStopping #regularization method to prevent the overfitting
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras import losses, optimizers
# Importing all the required sub-modules from Keras
from keras.applications.vgg16 import VGG16
from keras.preprocessing.image import ImageDataGenerator
from keras.preprocessing.image import img_to_array, load_img
# Mount Google drive to access the dataset
# Run the below code if you using google colab
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# Reading the dataset
images = np.load("/content/drive/MyDrive/images.npy")
labels = pd.read_csv("/content/drive/MyDrive/Labels.csv")
# Printing the images shape
images.shape
(4750, 128, 128, 3)
# Printing the labels shape
labels.shape
(4750, 1)
# Count in each category
label_counts = labels['Label'].value_counts(normalize=True)
print (label_counts)
Loose Silky-bent 0.137684 Common Chickweed 0.128632 Scentless Mayweed 0.108632 Small-flowered Cranesbill 0.104421 Fat Hen 0.100000 Charlock 0.082105 Sugar beet 0.081053 Cleavers 0.060421 Black-grass 0.055368 Shepherds Purse 0.048632 Common wheat 0.046526 Maize 0.046526 Name: Label, dtype: float64
# Plotting bar chart to see the data
label_counts.plot(kind='bar')
plt.title('Label Counts')
plt.xlabel('Label')
plt.ylabel('Count')
plt.show()
Let's visualize images randomly from each of the twelve classes.
# Generate random indices for each class
unique_labels = np.unique(labels)
num_classes = len(unique_labels)
# Generate random indices for each class
random_indices = []
for i in range(num_classes):
indices_for_class = np.where(labels == unique_labels[i])[0]
# Check if there are samples for the current class
if len(indices_for_class) > 0:
random_index = np.random.choice(indices_for_class)
random_indices.append(random_index)
else:
print(f"Class {i} has no samples.")
# Print the random indices for each class
for i, index in enumerate(random_indices):
print(f"Class {i}: Random Index {index}")
Class 0: Random Index 4059 Class 1: Random Index 2203 Class 2: Random Index 2457 Class 3: Random Index 1836 Class 4: Random Index 1214 Class 5: Random Index 636 Class 6: Random Index 4369 Class 7: Random Index 3771 Class 8: Random Index 2749 Class 9: Random Index 1016 Class 10: Random Index 28 Class 11: Random Index 3544
# Plot random images from each class
fig, axes = plt.subplots(4, 3, figsize=(15, 15))
axes = axes.flatten()
for i, index in enumerate(random_indices):
image = images[index]
label= labels.loc[index]
# Assuming images are in RGB format
axes[i].imshow(image)
axes[i].set_title(f'Class {label}')
axes[i].axis('off')
plt.show()
# Printing one individual image
plt.imshow(images[1010])
<matplotlib.image.AxesImage at 0x780ba149bd00>
# Convert the images to RGB from BGR
rgb_images = images[..., ::-1]
# Check the shape of the RGB images
rgb_images.shape
(4750, 128, 128, 3)
# Plot random RGB images from each class
fig, axes = plt.subplots(4, 3, figsize=(15, 15))
axes = axes.flatten()
for i, index in enumerate(random_indices):
image = rgb_images[index]
label= labels.loc[index]
# Assuming images are in RGB format
axes[i].imshow(image)
axes[i].set_title(f'Class {label}')
axes[i].axis('off')
plt.show()
# Plotting one RGB image
plt.imshow(rgb_images[1010])
<matplotlib.image.AxesImage at 0x780ba14995d0>
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
# First I trid to resize the images to 64 x 64 but did not get good result.
# so switched and resize the rgb images back to 128 x 128
resized_rgb_images = []
# Iterate through each image and resize
for image in rgb_images:
# Resize the image using OpenCV
resized_rgb_image = cv2.resize(image, (128, 128))
# Append the resized image to the list
resized_rgb_images.append(resized_rgb_image)
# Convert the list of resized images to a NumPy array
resized_rgb_images = np.array(resized_rgb_images)
# Check the images shape for resize
resized_rgb_images.shape
(4750, 128, 128, 3)
Split the dataset
# Let us train and test the
resized_rgb_images_2d = resized_rgb_images.reshape(resized_rgb_images.shape[0], 128,128,3)
print(resized_rgb_images_2d.shape)
print(labels.shape)
X_train, X_test, y_train, y_test = train_test_split(resized_rgb_images_2d
,labels , test_size=0.1, random_state=1,stratify=labels)
(4750, 128, 128, 3) (4750, 1)
# Converting the list into DataFrame
y_train = pd.DataFrame(y_train, columns=["Label"],dtype=object)
y_test = pd.DataFrame(y_test, columns=["Label"],dtype=object)
# Normalizing the image pixels
X_train = X_train.astype('float32')/255.0
X_test = X_test.astype('float32')/255.0
# Let us encode using Label Encoder
label_encoder = LabelEncoder()
# Fit the label encoder on the combined set of labels
all_labels = np.concatenate([y_train, y_test])
label_encoder.fit(all_labels)
# Transform labels to integers
y_train_e = label_encoder.transform(y_train)
y_test_e = label_encoder.transform(y_test)
/usr/local/lib/python3.10/dist-packages/sklearn/preprocessing/_label.py:99: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel(). y = column_or_1d(y, warn=True) /usr/local/lib/python3.10/dist-packages/sklearn/preprocessing/_label.py:134: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel(). y = column_or_1d(y, dtype=self.classes_.dtype, warn=True) /usr/local/lib/python3.10/dist-packages/sklearn/preprocessing/_label.py:134: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel(). y = column_or_1d(y, dtype=self.classes_.dtype, warn=True)
# Creating one-hot encoded representation of target labels
# We can do this by using this utility function - https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical
# to_categorical() function is also explained in the Neural Networks Module
y_train_encoded_oh = tf.keras.utils.to_categorical(y_train_e)
y_test_encoded_oh = tf.keras.utils.to_categorical(y_test_e)
y_train_e = tf.keras.utils.to_categorical(y_train_e, num_classes=12)
y_test_e = tf.keras.utils.to_categorical(y_test_e,num_classes=12)
Model 1 Limited Layers
from tensorflow.keras import backend
backend.clear_session()
np.random.seed(42)
import random
random.seed(42)
tf.random.set_seed(42)
# Initializing a sequential model
model = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3, padding 'same' provides the output size
# same as the input size. Input_shape denotes input image dimension of MNIST images
model.add(Conv2D(64,(3,3), activation='relu',padding='same', input_shape=(128,128,3)))
# Adding max pooling to reduce the size of output of first conv layer
model.add(MaxPooling2D((2,2),padding='same'))
model.add(Conv2D(32,(3,3), activation='relu', padding='same'))
model.add(MaxPooling2D((2,2),padding='same'))
model.add(Conv2D(32,(3,3),activation='relu',padding='same'))
model.add(MaxPooling2D((2,2),padding='same'))
# Flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model.add(Flatten())
# Adding a fully connected dense layerwith 100 neurons
model.add(Dense(100,activation='relu'))
# Adding the output layer with 10 neurons and activation functions as softmax since this is multi-calssification problem
model.add(Dense(12,activation='softmax'))
#Using SDG Optimizer
opt = SGD(learning_rate=0.01,momentum=0.9)
# Compile Model
model.compile(optimizer=opt, loss='categorical_crossentropy',metrics=['accuracy'])
# Generating the summary of the model
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 128, 128, 64) 1792
max_pooling2d (MaxPooling2 (None, 64, 64, 64) 0
D)
conv2d_1 (Conv2D) (None, 64, 64, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 32, 32, 32) 0
g2D)
conv2d_2 (Conv2D) (None, 32, 32, 32) 9248
max_pooling2d_2 (MaxPoolin (None, 16, 16, 32) 0
g2D)
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 100) 819300
dense_1 (Dense) (None, 12) 1212
=================================================================
Total params: 850016 (3.24 MB)
Trainable params: 850016 (3.24 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Let us fit the model on the training data
from keras.callbacks import EarlyStopping, ModelCheckpoint
es = EarlyStopping(monitor='val_loss',mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.h5',monitor='val_accuracy',mode='max',verbose=1, save_best_only=True)
# Fitting the model with 30 epochs and validation_split as 10%
history=model.fit(X_train,
y_train_e,
epochs=30,
batch_size=32,validation_split=0.10,callbacks=[es,mc])
Epoch 1/30 119/121 [============================>.] - ETA: 0s - loss: 2.4341 - accuracy: 0.1444 Epoch 1: val_accuracy improved from -inf to 0.17523, saving model to best_model.h5 121/121 [==============================] - 4s 24ms/step - loss: 2.4336 - accuracy: 0.1435 - val_loss: 2.3700 - val_accuracy: 0.1752 Epoch 2/30 7/121 [>.............................] - ETA: 2s - loss: 2.4220 - accuracy: 0.1696
/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3079: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
119/121 [============================>.] - ETA: 0s - loss: 2.2287 - accuracy: 0.2379 Epoch 2: val_accuracy improved from 0.17523 to 0.41121, saving model to best_model.h5 121/121 [==============================] - 3s 23ms/step - loss: 2.2250 - accuracy: 0.2389 - val_loss: 1.7588 - val_accuracy: 0.4112 Epoch 3/30 121/121 [==============================] - ETA: 0s - loss: 1.5070 - accuracy: 0.4835 Epoch 3: val_accuracy improved from 0.41121 to 0.60981, saving model to best_model.h5 121/121 [==============================] - 3s 21ms/step - loss: 1.5070 - accuracy: 0.4835 - val_loss: 1.2572 - val_accuracy: 0.6098 Epoch 4/30 121/121 [==============================] - ETA: 0s - loss: 1.1572 - accuracy: 0.6020 Epoch 4: val_accuracy improved from 0.60981 to 0.63785, saving model to best_model.h5 121/121 [==============================] - 3s 23ms/step - loss: 1.1572 - accuracy: 0.6020 - val_loss: 1.1351 - val_accuracy: 0.6379 Epoch 5/30 119/121 [============================>.] - ETA: 0s - loss: 0.9913 - accuracy: 0.6570 Epoch 5: val_accuracy did not improve from 0.63785 121/121 [==============================] - 3s 23ms/step - loss: 0.9913 - accuracy: 0.6571 - val_loss: 1.4791 - val_accuracy: 0.5561 Epoch 6/30 120/121 [============================>.] - ETA: 0s - loss: 0.8363 - accuracy: 0.7122 Epoch 6: val_accuracy improved from 0.63785 to 0.67991, saving model to best_model.h5 121/121 [==============================] - 3s 22ms/step - loss: 0.8362 - accuracy: 0.7122 - val_loss: 1.0192 - val_accuracy: 0.6799 Epoch 7/30 118/121 [============================>.] - ETA: 0s - loss: 0.6619 - accuracy: 0.7712 Epoch 7: val_accuracy improved from 0.67991 to 0.69393, saving model to best_model.h5 121/121 [==============================] - 2s 20ms/step - loss: 0.6644 - accuracy: 0.7697 - val_loss: 1.0967 - val_accuracy: 0.6939 Epoch 8/30 118/121 [============================>.] - ETA: 0s - loss: 0.5723 - accuracy: 0.7977 Epoch 8: val_accuracy improved from 0.69393 to 0.71262, saving model to best_model.h5 121/121 [==============================] - 2s 20ms/step - loss: 0.5732 - accuracy: 0.7988 - val_loss: 0.9135 - val_accuracy: 0.7126 Epoch 9/30 118/121 [============================>.] - ETA: 0s - loss: 0.4348 - accuracy: 0.8477 Epoch 9: val_accuracy improved from 0.71262 to 0.77570, saving model to best_model.h5 121/121 [==============================] - 2s 20ms/step - loss: 0.4401 - accuracy: 0.8456 - val_loss: 0.8740 - val_accuracy: 0.7757 Epoch 10/30 120/121 [============================>.] - ETA: 0s - loss: 0.3573 - accuracy: 0.8690 Epoch 10: val_accuracy did not improve from 0.77570 121/121 [==============================] - 3s 21ms/step - loss: 0.3577 - accuracy: 0.8690 - val_loss: 0.9658 - val_accuracy: 0.7033 Epoch 11/30 120/121 [============================>.] - ETA: 0s - loss: 0.3361 - accuracy: 0.8836 Epoch 11: val_accuracy did not improve from 0.77570 121/121 [==============================] - 3s 23ms/step - loss: 0.3361 - accuracy: 0.8835 - val_loss: 1.0867 - val_accuracy: 0.7336 Epoch 12/30 119/121 [============================>.] - ETA: 0s - loss: 0.2984 - accuracy: 0.8936 Epoch 12: val_accuracy did not improve from 0.77570 121/121 [==============================] - 3s 22ms/step - loss: 0.2974 - accuracy: 0.8939 - val_loss: 0.8773 - val_accuracy: 0.7664 Epoch 13/30 118/121 [============================>.] - ETA: 0s - loss: 0.1374 - accuracy: 0.9523 Epoch 13: val_accuracy did not improve from 0.77570 121/121 [==============================] - 2s 21ms/step - loss: 0.1374 - accuracy: 0.9519 - val_loss: 1.1081 - val_accuracy: 0.7570 Epoch 14/30 120/121 [============================>.] - ETA: 0s - loss: 0.1247 - accuracy: 0.9552 Epoch 14: val_accuracy did not improve from 0.77570 121/121 [==============================] - 2s 21ms/step - loss: 0.1245 - accuracy: 0.9553 - val_loss: 1.2681 - val_accuracy: 0.7734 Epoch 14: early stopping
Plotting Accuracy vs Epoch curve
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model_accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train','test'],loc='upper left')
plt.show()
model.evaluate(X_test,(y_test_e))
15/15 [==============================] - 0s 9ms/step - loss: 1.2502 - accuracy: 0.7453
[1.2502117156982422, 0.7452631592750549]
# Test Prediction
y_test_pred_ln = model.predict(X_test)
y_test_pred_classes_ln = np.argmax(y_test_pred_ln, axis=1)
normal_y_test = np.argmax(y_test_e, axis=1)
15/15 [==============================] - 0s 9ms/step
# Test Accuracy
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score((normal_y_test),y_test_pred_classes_ln)
0.7452631578947368
# Let us print the sns heatmap
CATEGORIES = list(unique_labels)
cf_matrix = confusion_matrix(normal_y_test, y_test_pred_classes_ln)
# Confusion matrix normalized per category true value
cf_matrix_n1 = cf_matrix/np.sum(cf_matrix, axis=1)
plt.figure(figsize=(8,6))
sns.heatmap(cf_matrix_n1, xticklabels=CATEGORIES, yticklabels=CATEGORIES, annot=True)
<Axes: >
Observations:
Let try to build another CNN model with more layers added to the model
from tensorflow.keras import backend
backend.clear_session()
#Fixing the seed for random number generators so that we can ensure we receive the same output everytime
np.random.seed(42)
import random
random.seed(42)
tf.random.set_seed(42)
# initialized a sequential model
model_3 = Sequential()
# adding first conv layer with 256 filters and kernel size 5x5 , with ReLU activation and padding 'same' provides the output size same as the input size
#input_shape denotes input image dimension of images
model_3.add(Conv2D(filters = 256, kernel_size = (3,3),padding = 'Same',
activation ='relu', input_shape = (128,128,3)))
# adding max pooling to reduce the size of output of first conv layer
model_3.add(MaxPool2D(pool_size=(2,2)))
# adding dropout to randomly switch off 25% neurons to reduce overfitting
# adding second conv layer with 256 filters and with kernel size 3x3 and ReLu activation function
model_3.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
# adding max pooling to reduce the size of output of first conv layer
model_3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# adding dropout to randomly switch off 25% neurons to reduce overfitting
# adding third conv layer with 256 filters and with kernel size 3x3 and ReLu activation function
model_3.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
# adding max pooling to reduce the size of output of first conv layer
model_3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# adding dropout to randomly switch off 30% neurons to reduce overfitting
model_3.add(Dropout(0.3))
# adding forth conv layer with 256 filters and with kernel size 3x3 and ReLu activation function
model_3.add(Conv2D(filters = 32, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
# adding max pooling to reduce the size of output of first conv layer
model_3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
# adding dropout to randomly switch off 30% neurons to reduce overfitting
model_3.add(Dropout(0.3))
# flattening the 3-d output of the conv layer after max pooling to make it ready for creating dense connections
model_3.add(Flatten())
# adding first fully connected dense layer with 1024 neurons
model_3.add(Dense(64, activation = "relu"))
# adding dropout to randomly switch off 50% neurons to reduce overfitting
# model_3.add(Dropout(0.5))
# adding second fully connected dense layer with 512 neurons
model_3.add(Dense(32, activation = "relu"))
# adding dropout to randomly switch off 50% neurons to reduce overfitting
#model_3.add(Dropout(0.5))
# adding the output layer with 3 neurons and activation functions as softmax since this is a multi-class classification problem with 3 classes.
model_3.add(Dense(12, activation = "softmax"))
# Let us print the model summary
model_3.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 128, 128, 256) 7168
max_pooling2d (MaxPooling2 (None, 64, 64, 256) 0
D)
conv2d_1 (Conv2D) (None, 64, 64, 128) 295040
max_pooling2d_1 (MaxPoolin (None, 32, 32, 128) 0
g2D)
conv2d_2 (Conv2D) (None, 32, 32, 64) 73792
max_pooling2d_2 (MaxPoolin (None, 16, 16, 64) 0
g2D)
dropout (Dropout) (None, 16, 16, 64) 0
conv2d_3 (Conv2D) (None, 16, 16, 32) 18464
max_pooling2d_3 (MaxPoolin (None, 8, 8, 32) 0
g2D)
dropout_1 (Dropout) (None, 8, 8, 32) 0
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 64) 131136
dense_1 (Dense) (None, 32) 2080
dense_2 (Dense) (None, 12) 396
=================================================================
Total params: 528076 (2.01 MB)
Trainable params: 528076 (2.01 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Let us defind the early stopping, ModelCheckPoint variables and fit the mode on the training data
optimizer = Adam(lr=0.001)
model_3.compile(optimizer = optimizer , loss = "categorical_crossentropy", metrics=["accuracy"])
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=8)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
history=model_3.fit(X_train,
y_train_e,
epochs=30,
batch_size=32,validation_split=0.10,callbacks=[es, mc],use_multiprocessing=True)
WARNING:absl:`lr` is deprecated in Keras optimizer, please use `learning_rate` or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
Epoch 1/30 120/121 [============================>.] - ETA: 0s - loss: 2.4443 - accuracy: 0.1237 Epoch 1: val_accuracy improved from -inf to 0.16121, saving model to best_model.h5 121/121 [==============================] - 16s 108ms/step - loss: 2.4438 - accuracy: 0.1243 - val_loss: 2.3810 - val_accuracy: 0.1612 Epoch 2/30 1/121 [..............................] - ETA: 11s - loss: 2.5613 - accuracy: 0.0312
/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3079: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
120/121 [============================>.] - ETA: 0s - loss: 2.4271 - accuracy: 0.1318 Epoch 2: val_accuracy did not improve from 0.16121 121/121 [==============================] - 12s 99ms/step - loss: 2.4271 - accuracy: 0.1315 - val_loss: 2.4086 - val_accuracy: 0.1355 Epoch 3/30 120/121 [============================>.] - ETA: 0s - loss: 2.4202 - accuracy: 0.1391 Epoch 3: val_accuracy did not improve from 0.16121 121/121 [==============================] - 12s 100ms/step - loss: 2.4204 - accuracy: 0.1388 - val_loss: 2.3821 - val_accuracy: 0.1472 Epoch 4/30 120/121 [============================>.] - ETA: 0s - loss: 1.9083 - accuracy: 0.3570 Epoch 4: val_accuracy improved from 0.16121 to 0.39953, saving model to best_model.h5 121/121 [==============================] - 12s 99ms/step - loss: 1.9061 - accuracy: 0.3579 - val_loss: 1.6946 - val_accuracy: 0.3995 Epoch 5/30 120/121 [============================>.] - ETA: 0s - loss: 1.4454 - accuracy: 0.4862 Epoch 5: val_accuracy improved from 0.39953 to 0.60748, saving model to best_model.h5 121/121 [==============================] - 12s 95ms/step - loss: 1.4454 - accuracy: 0.4866 - val_loss: 1.1850 - val_accuracy: 0.6075 Epoch 6/30 120/121 [============================>.] - ETA: 0s - loss: 1.1217 - accuracy: 0.6112 Epoch 6: val_accuracy improved from 0.60748 to 0.66121, saving model to best_model.h5 121/121 [==============================] - 12s 95ms/step - loss: 1.1222 - accuracy: 0.6111 - val_loss: 1.1157 - val_accuracy: 0.6612 Epoch 7/30 120/121 [============================>.] - ETA: 0s - loss: 0.9611 - accuracy: 0.6677 Epoch 7: val_accuracy improved from 0.66121 to 0.74065, saving model to best_model.h5 121/121 [==============================] - 12s 96ms/step - loss: 0.9609 - accuracy: 0.6675 - val_loss: 0.9568 - val_accuracy: 0.7407 Epoch 8/30 120/121 [============================>.] - ETA: 0s - loss: 0.8706 - accuracy: 0.7023 Epoch 8: val_accuracy did not improve from 0.74065 121/121 [==============================] - 11s 95ms/step - loss: 0.8702 - accuracy: 0.7018 - val_loss: 0.8355 - val_accuracy: 0.7360 Epoch 9/30 120/121 [============================>.] - ETA: 0s - loss: 0.7484 - accuracy: 0.7378 Epoch 9: val_accuracy did not improve from 0.74065 121/121 [==============================] - 11s 95ms/step - loss: 0.7481 - accuracy: 0.7377 - val_loss: 0.9552 - val_accuracy: 0.7126 Epoch 10/30 120/121 [============================>.] - ETA: 0s - loss: 0.6971 - accuracy: 0.7578 Epoch 10: val_accuracy improved from 0.74065 to 0.79206, saving model to best_model.h5 121/121 [==============================] - 12s 96ms/step - loss: 0.6968 - accuracy: 0.7580 - val_loss: 0.7605 - val_accuracy: 0.7921 Epoch 11/30 120/121 [============================>.] - ETA: 0s - loss: 0.6311 - accuracy: 0.7789 Epoch 11: val_accuracy did not improve from 0.79206 121/121 [==============================] - 12s 96ms/step - loss: 0.6304 - accuracy: 0.7793 - val_loss: 0.7351 - val_accuracy: 0.7780 Epoch 12/30 120/121 [============================>.] - ETA: 0s - loss: 0.5713 - accuracy: 0.8055 Epoch 12: val_accuracy improved from 0.79206 to 0.82710, saving model to best_model.h5 121/121 [==============================] - 11s 95ms/step - loss: 0.5716 - accuracy: 0.8053 - val_loss: 0.6542 - val_accuracy: 0.8271 Epoch 13/30 120/121 [============================>.] - ETA: 0s - loss: 0.5302 - accuracy: 0.8096 Epoch 13: val_accuracy did not improve from 0.82710 121/121 [==============================] - 12s 96ms/step - loss: 0.5312 - accuracy: 0.8092 - val_loss: 0.6726 - val_accuracy: 0.7991 Epoch 14/30 120/121 [============================>.] - ETA: 0s - loss: 0.4717 - accuracy: 0.8271 Epoch 14: val_accuracy did not improve from 0.82710 121/121 [==============================] - 12s 95ms/step - loss: 0.4716 - accuracy: 0.8271 - val_loss: 0.6769 - val_accuracy: 0.8014 Epoch 15/30 120/121 [============================>.] - ETA: 0s - loss: 0.4376 - accuracy: 0.8469 Epoch 15: val_accuracy did not improve from 0.82710 121/121 [==============================] - 11s 95ms/step - loss: 0.4371 - accuracy: 0.8472 - val_loss: 0.7460 - val_accuracy: 0.7874 Epoch 16/30 120/121 [============================>.] - ETA: 0s - loss: 0.3894 - accuracy: 0.8646 Epoch 16: val_accuracy did not improve from 0.82710 121/121 [==============================] - 12s 95ms/step - loss: 0.3890 - accuracy: 0.8648 - val_loss: 0.7398 - val_accuracy: 0.7827 Epoch 17/30 120/121 [============================>.] - ETA: 0s - loss: 0.3818 - accuracy: 0.8648 Epoch 17: val_accuracy did not improve from 0.82710 121/121 [==============================] - 12s 96ms/step - loss: 0.3821 - accuracy: 0.8646 - val_loss: 0.7657 - val_accuracy: 0.8014 Epoch 18/30 120/121 [============================>.] - ETA: 0s - loss: 0.3512 - accuracy: 0.8792 Epoch 18: val_accuracy did not improve from 0.82710 121/121 [==============================] - 11s 94ms/step - loss: 0.3513 - accuracy: 0.8791 - val_loss: 0.7402 - val_accuracy: 0.8084 Epoch 19/30 120/121 [============================>.] - ETA: 0s - loss: 0.3337 - accuracy: 0.8784 Epoch 19: val_accuracy did not improve from 0.82710 121/121 [==============================] - 11s 94ms/step - loss: 0.3344 - accuracy: 0.8781 - val_loss: 0.7441 - val_accuracy: 0.8131 Epoch 20/30 120/121 [============================>.] - ETA: 0s - loss: 0.3093 - accuracy: 0.8875 Epoch 20: val_accuracy did not improve from 0.82710 121/121 [==============================] - 12s 96ms/step - loss: 0.3089 - accuracy: 0.8877 - val_loss: 0.7785 - val_accuracy: 0.7804 Epoch 20: early stopping
Plotting Accuracy vs Epoch Curve
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'Val'], loc='upper left')
plt.show()
Train and Validation accuracy seems fine and let's calculate the accuracy on the test data
model_3.evaluate(X_test,y_test_e)
15/15 [==============================] - 0s 24ms/step - loss: 0.8055 - accuracy: 0.7768
[0.8055254220962524, 0.7768421173095703]
y_test_pred_ln3 = model_3.predict(X_test)
y_test_pred_classes_ln3 = np.argmax(y_test_pred_ln3, axis=1)
15/15 [==============================] - 0s 17ms/step
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score(normal_y_test, y_test_pred_classes_ln3)
0.7768421052631579
cf_matrix = confusion_matrix(normal_y_test, y_test_pred_classes_ln3)
# Confusion matrix normalized per category true value
cf_matrix_n1 = cf_matrix/np.sum(cf_matrix, axis=1)
plt.figure(figsize=(8,6))
sns.heatmap(cf_matrix_n1, xticklabels=CATEGORIES, yticklabels=CATEGORIES, annot=True)
<Axes: >
Observations:
Let us try with Data Agumentation
Displaying images before augmentation
num_images = 5
plt.figure(figsize=(15, 15))
for i in range(num_images):
plt.subplot(1, num_images, i + 1)
plt.imshow(X_train[i])
plt.axis('off')
plt.show()
# All images to be rescaled by 1/255.
train_datagen = ImageDataGenerator(
rescale=1./255, # normalize pixel values to [0,1]
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
)
test_datagen = ImageDataGenerator(rescale = 1.0/255.)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
# Fit the generators on the training data
train_datagen.fit(X_train)
# Generate augmented images for training
train_generator = train_datagen.flow(X_train, y_train_e, batch_size=24,
shuffle=True)
# Generate rescaled images for testing (without augmentation)
test_generator = test_datagen.flow(X_test, y_test_e, batch_size=24,
shuffle=False)
# Number of images to display
num_images = 5
# Obtain a batch of augmented images and labels from the generator
augmented_images, augmented_labels = train_generator.next()
# Display the images
plt.figure(figsize=(15, 15))
for i in range(num_images):
plt.subplot(1, num_images, i + 1)
plt.imshow(augmented_images[i]*255)
plt.axis('off')
plt.show()
Let us build the model
cnn_model = Sequential()
cnn_model.add(Conv2D(128, (3,3), activation='relu', input_shape=(128, 128, 3), padding = 'same'))
cnn_model.add(MaxPooling2D(2,2))
cnn_model.add(BatchNormalization())
cnn_model.add(Conv2D(64, (3,3), activation='relu', padding = 'same'))
cnn_model.add(MaxPooling2D(2,2))
cnn_model.add(BatchNormalization())
cnn_model.add(Conv2D(64, (3,3), activation='relu', padding = 'same'))
cnn_model.add(MaxPooling2D(2,2))
cnn_model.add(Conv2D(16, (3,3), activation='relu', padding = 'same'))
cnn_model.add(Flatten())
cnn_model.add(Dense(64, activation='relu'))
cnn_model.add(Dropout(0.25))
cnn_model.add(Dense(32, activation='relu'))
cnn_model.add(Dropout(0.25))
cnn_model.add(Dense(32, activation='relu'))
cnn_model.add(Dense(12, activation='softmax'))
# Compiling the model
cnn_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Summary of the model
cnn_model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_4 (Conv2D) (None, 128, 128, 128) 3584
max_pooling2d_4 (MaxPoolin (None, 64, 64, 128) 0
g2D)
batch_normalization (Batch (None, 64, 64, 128) 512
Normalization)
conv2d_5 (Conv2D) (None, 64, 64, 64) 73792
max_pooling2d_5 (MaxPoolin (None, 32, 32, 64) 0
g2D)
batch_normalization_1 (Bat (None, 32, 32, 64) 256
chNormalization)
conv2d_6 (Conv2D) (None, 32, 32, 64) 36928
max_pooling2d_6 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_7 (Conv2D) (None, 16, 16, 16) 9232
flatten_1 (Flatten) (None, 4096) 0
dense_3 (Dense) (None, 64) 262208
dropout_2 (Dropout) (None, 64) 0
dense_4 (Dense) (None, 32) 2080
dropout_3 (Dropout) (None, 32) 0
dense_5 (Dense) (None, 32) 1056
dense_6 (Dense) (None, 12) 396
=================================================================
Total params: 390044 (1.49 MB)
Trainable params: 389660 (1.49 MB)
Non-trainable params: 384 (1.50 KB)
_________________________________________________________________
## Pulling a single large batch of random validation data for testing after each epoch
testX, testY = test_generator.next()
Let us fit the model with augmented data and check the performance
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
## Fitting the Aug model
cnn_model_history = cnn_model.fit(train_generator,
validation_data = (testX, testY),
epochs=30,callbacks=[es, mc],use_multiprocessing=True)
Epoch 1/30 179/179 [==============================] - ETA: 0s - loss: 2.3233 - accuracy: 0.1977 Epoch 1: val_accuracy improved from -inf to 0.12500, saving model to best_model.h5 179/179 [==============================] - 44s 215ms/step - loss: 2.3233 - accuracy: 0.1977 - val_loss: 2.6576 - val_accuracy: 0.1250
/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3079: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
saving_api.save_model(
Epoch 2/30 179/179 [==============================] - ETA: 0s - loss: 1.9627 - accuracy: 0.2924 Epoch 2: val_accuracy did not improve from 0.12500 179/179 [==============================] - 44s 242ms/step - loss: 1.9627 - accuracy: 0.2924 - val_loss: 4.6108 - val_accuracy: 0.1250 Epoch 3/30 179/179 [==============================] - ETA: 0s - loss: 1.8166 - accuracy: 0.3570 Epoch 3: val_accuracy did not improve from 0.12500 179/179 [==============================] - 43s 241ms/step - loss: 1.8166 - accuracy: 0.3570 - val_loss: 2.3390 - val_accuracy: 0.1250 Epoch 4/30 179/179 [==============================] - ETA: 0s - loss: 1.6662 - accuracy: 0.4091 Epoch 4: val_accuracy improved from 0.12500 to 0.16667, saving model to best_model.h5 179/179 [==============================] - 45s 251ms/step - loss: 1.6662 - accuracy: 0.4091 - val_loss: 2.1631 - val_accuracy: 0.1667 Epoch 5/30 179/179 [==============================] - ETA: 0s - loss: 1.5827 - accuracy: 0.4374 Epoch 5: val_accuracy did not improve from 0.16667 179/179 [==============================] - 44s 245ms/step - loss: 1.5827 - accuracy: 0.4374 - val_loss: 2.2272 - val_accuracy: 0.1250 Epoch 6/30 179/179 [==============================] - ETA: 0s - loss: 1.5229 - accuracy: 0.4643 Epoch 6: val_accuracy did not improve from 0.16667 179/179 [==============================] - 42s 235ms/step - loss: 1.5229 - accuracy: 0.4643 - val_loss: 7.0037 - val_accuracy: 0.0000e+00 Epoch 7/30 179/179 [==============================] - ETA: 0s - loss: 1.4590 - accuracy: 0.4936 Epoch 7: val_accuracy improved from 0.16667 to 0.54167, saving model to best_model.h5 179/179 [==============================] - 46s 253ms/step - loss: 1.4590 - accuracy: 0.4936 - val_loss: 1.4965 - val_accuracy: 0.5417 Epoch 8/30 179/179 [==============================] - ETA: 0s - loss: 1.3922 - accuracy: 0.4971 Epoch 8: val_accuracy did not improve from 0.54167 179/179 [==============================] - 45s 249ms/step - loss: 1.3922 - accuracy: 0.4971 - val_loss: 1.4369 - val_accuracy: 0.5417 Epoch 9/30 179/179 [==============================] - ETA: 0s - loss: 1.3745 - accuracy: 0.5268 Epoch 9: val_accuracy did not improve from 0.54167 179/179 [==============================] - 45s 249ms/step - loss: 1.3745 - accuracy: 0.5268 - val_loss: 3.6982 - val_accuracy: 0.1667 Epoch 10/30 179/179 [==============================] - ETA: 0s - loss: 1.3085 - accuracy: 0.5427 Epoch 10: val_accuracy improved from 0.54167 to 0.62500, saving model to best_model.h5 179/179 [==============================] - 47s 262ms/step - loss: 1.3085 - accuracy: 0.5427 - val_loss: 1.1831 - val_accuracy: 0.6250 Epoch 11/30 179/179 [==============================] - ETA: 0s - loss: 1.2812 - accuracy: 0.5593 Epoch 11: val_accuracy did not improve from 0.62500 179/179 [==============================] - 43s 238ms/step - loss: 1.2812 - accuracy: 0.5593 - val_loss: 2.1719 - val_accuracy: 0.2500 Epoch 12/30 179/179 [==============================] - ETA: 0s - loss: 1.2428 - accuracy: 0.5680 Epoch 12: val_accuracy did not improve from 0.62500 179/179 [==============================] - 43s 239ms/step - loss: 1.2428 - accuracy: 0.5680 - val_loss: 1.3689 - val_accuracy: 0.5833 Epoch 13/30 179/179 [==============================] - ETA: 0s - loss: 1.2131 - accuracy: 0.5726 Epoch 13: val_accuracy did not improve from 0.62500 179/179 [==============================] - 47s 261ms/step - loss: 1.2131 - accuracy: 0.5726 - val_loss: 1.3967 - val_accuracy: 0.6250 Epoch 14/30 179/179 [==============================] - ETA: 0s - loss: 1.1851 - accuracy: 0.5937 Epoch 14: val_accuracy did not improve from 0.62500 179/179 [==============================] - 43s 240ms/step - loss: 1.1851 - accuracy: 0.5937 - val_loss: 3.3549 - val_accuracy: 0.1250 Epoch 15/30 179/179 [==============================] - ETA: 0s - loss: 1.2075 - accuracy: 0.5780 Epoch 15: val_accuracy did not improve from 0.62500 179/179 [==============================] - 44s 244ms/step - loss: 1.2075 - accuracy: 0.5780 - val_loss: 1.3158 - val_accuracy: 0.5417 Epoch 15: early stopping
Observation:
# Loading VGG16 model
model = VGG16(weights='imagenet')
# Summary of the whole model
model.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 224, 224, 3)] 0
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
flatten (Flatten) (None, 25088) 0
fc1 (Dense) (None, 4096) 102764544
fc2 (Dense) (None, 4096) 16781312
predictions (Dense) (None, 1000) 4097000
=================================================================
Total params: 138357544 (527.79 MB)
Trainable params: 138357544 (527.79 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Getting only the conv layers for transfer learning.
transfer_layer = model.get_layer('block5_pool')
vgg_model = Model(inputs=model.input, outputs=transfer_layer.output)
# Printing the model summary
vgg_model.summary()
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 224, 224, 3)] 0
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (128,128,3))
vgg_model.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 128, 128, 3)] 0
block1_conv1 (Conv2D) (None, 128, 128, 64) 1792
block1_conv2 (Conv2D) (None, 128, 128, 64) 36928
block1_pool (MaxPooling2D) (None, 64, 64, 64) 0
block2_conv1 (Conv2D) (None, 64, 64, 128) 73856
block2_conv2 (Conv2D) (None, 64, 64, 128) 147584
block2_pool (MaxPooling2D) (None, 32, 32, 128) 0
block3_conv1 (Conv2D) (None, 32, 32, 256) 295168
block3_conv2 (Conv2D) (None, 32, 32, 256) 590080
block3_conv3 (Conv2D) (None, 32, 32, 256) 590080
block3_pool (MaxPooling2D) (None, 16, 16, 256) 0
block4_conv1 (Conv2D) (None, 16, 16, 512) 1180160
block4_conv2 (Conv2D) (None, 16, 16, 512) 2359808
block4_conv3 (Conv2D) (None, 16, 16, 512) 2359808
block4_pool (MaxPooling2D) (None, 8, 8, 512) 0
block5_conv1 (Conv2D) (None, 8, 8, 512) 2359808
block5_conv2 (Conv2D) (None, 8, 8, 512) 2359808
block5_conv3 (Conv2D) (None, 8, 8, 512) 2359808
block5_pool (MaxPooling2D) (None, 4, 4, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
layer.trainable = False
for layer in vgg_model.layers:
print(layer.name, layer.trainable)
input_2 False block1_conv1 False block1_conv2 False block1_pool False block2_conv1 False block2_conv2 False block2_pool False block3_conv1 False block3_conv2 False block3_conv3 False block3_pool False block4_conv1 False block4_conv2 False block4_conv3 False block4_pool False block5_conv1 False block5_conv2 False block5_conv3 False block5_pool False
Let us start building the model
backend.clear_session()
#Fixing the seed for random number generators so that we can ensure we receive the same output everytime
np.random.seed(42)
import random
random.seed(42)
tf.random.set_seed(42)
# Initializing the model
new_model = Sequential()
# Adding the convolutional part of the VGG16 model from above
new_model.add(vgg_model)
# Flattening the output of the VGG16 model because it is from a convolutional layer
new_model.add(Flatten())
# Adding a dense input layer
new_model.add(Dense(128, activation='relu'))
# Adding dropout
new_model.add(Dropout(0.2))
# Adding second input layer
new_model.add(Dense(128, activation='relu'))
# Adding output layer
new_model.add(Dense(12, activation='softmax'))
# Compiling the model
new_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Summary of the model
new_model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 4, 4, 512) 14714688
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 128) 1048704
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 128) 16512
dense_2 (Dense) (None, 12) 1548
=================================================================
Total params: 15781452 (60.20 MB)
Trainable params: 1066764 (4.07 MB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
## Pulling a single large batch of random validation data for testing after each epoch
testX, testY = test_generator.next()
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
## Fitting the VGG model
#new_model_history = new_model.fit(train_generator,
# validation_data = (testX, testY),
# epochs=30,callbacks=[es, mc],use_multiprocessing=True)
new_model_history=new_model.fit(X_train,
y_train_e,
epochs=30,
batch_size=32,validation_split=0.10,callbacks=[es, mc],use_multiprocessing=True)
Epoch 1/30 120/121 [============================>.] - ETA: 0s - loss: 1.9156 - accuracy: 0.3484 Epoch 1: val_accuracy improved from -inf to 0.44626, saving model to best_model.h5 121/121 [==============================] - 9s 55ms/step - loss: 1.9150 - accuracy: 0.3486 - val_loss: 1.5390 - val_accuracy: 0.4463 Epoch 2/30 121/121 [==============================] - ETA: 0s - loss: 1.3724 - accuracy: 0.5157 Epoch 2: val_accuracy improved from 0.44626 to 0.55374, saving model to best_model.h5 121/121 [==============================] - 7s 55ms/step - loss: 1.3724 - accuracy: 0.5157 - val_loss: 1.2697 - val_accuracy: 0.5537 Epoch 3/30 120/121 [============================>.] - ETA: 0s - loss: 1.0423 - accuracy: 0.6393 Epoch 3: val_accuracy improved from 0.55374 to 0.69860, saving model to best_model.h5 121/121 [==============================] - 7s 54ms/step - loss: 1.0414 - accuracy: 0.6397 - val_loss: 0.9807 - val_accuracy: 0.6986 Epoch 4/30 121/121 [==============================] - ETA: 0s - loss: 0.9000 - accuracy: 0.6914 Epoch 4: val_accuracy improved from 0.69860 to 0.70327, saving model to best_model.h5 121/121 [==============================] - 7s 54ms/step - loss: 0.9000 - accuracy: 0.6914 - val_loss: 0.9205 - val_accuracy: 0.7033 Epoch 5/30 120/121 [============================>.] - ETA: 0s - loss: 0.7580 - accuracy: 0.7354 Epoch 5: val_accuracy improved from 0.70327 to 0.72196, saving model to best_model.h5 121/121 [==============================] - 6s 54ms/step - loss: 0.7578 - accuracy: 0.7356 - val_loss: 0.8349 - val_accuracy: 0.7220 Epoch 6/30 120/121 [============================>.] - ETA: 0s - loss: 0.6972 - accuracy: 0.7542 Epoch 6: val_accuracy improved from 0.72196 to 0.76168, saving model to best_model.h5 121/121 [==============================] - 6s 52ms/step - loss: 0.6968 - accuracy: 0.7544 - val_loss: 0.8056 - val_accuracy: 0.7617 Epoch 7/30 120/121 [============================>.] - ETA: 0s - loss: 0.5997 - accuracy: 0.7888 Epoch 7: val_accuracy did not improve from 0.76168 121/121 [==============================] - 6s 51ms/step - loss: 0.5995 - accuracy: 0.7887 - val_loss: 0.8200 - val_accuracy: 0.7383 Epoch 8/30 120/121 [============================>.] - ETA: 0s - loss: 0.5646 - accuracy: 0.8036 Epoch 8: val_accuracy did not improve from 0.76168 121/121 [==============================] - 6s 50ms/step - loss: 0.5639 - accuracy: 0.8040 - val_loss: 0.8450 - val_accuracy: 0.7290 Epoch 9/30 121/121 [==============================] - ETA: 0s - loss: 0.5178 - accuracy: 0.8162 Epoch 9: val_accuracy did not improve from 0.76168 121/121 [==============================] - 6s 49ms/step - loss: 0.5178 - accuracy: 0.8162 - val_loss: 0.8209 - val_accuracy: 0.7313 Epoch 10/30 120/121 [============================>.] - ETA: 0s - loss: 0.4687 - accuracy: 0.8378 Epoch 10: val_accuracy did not improve from 0.76168 121/121 [==============================] - 6s 51ms/step - loss: 0.4687 - accuracy: 0.8378 - val_loss: 0.8398 - val_accuracy: 0.7336 Epoch 11/30 120/121 [============================>.] - ETA: 0s - loss: 0.4576 - accuracy: 0.8393 Epoch 11: val_accuracy improved from 0.76168 to 0.76402, saving model to best_model.h5 121/121 [==============================] - 6s 51ms/step - loss: 0.4579 - accuracy: 0.8391 - val_loss: 0.8141 - val_accuracy: 0.7640 Epoch 11: early stopping
Plotting Accuracy vs Epoch Curve
plt.plot(new_model_history.history['accuracy'])
plt.plot(new_model_history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'Val'], loc='upper left')
plt.show()
# Evaluating on the Test set
new_model.evaluate(test_generator)
20/20 [==============================] - 1s 40ms/step - loss: 5.0914 - accuracy: 0.1368
[5.091447830200195, 0.13684210181236267]
# Test Prediction
y_test_pred_ln4 = new_model.predict(X_test)
y_test_pred_classes_ln4 = np.argmax(y_test_pred_ln4, axis=1)
15/15 [==============================] - 1s 41ms/step
# Test Accuracy
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score(normal_y_test, y_test_pred_classes_ln4)
0.7326315789473684
# Let us print the sns heatmap
cf_matrix = confusion_matrix(normal_y_test, y_test_pred_classes_ln4)
# Confusion matrix normalized per category true value
cf_matrix_n1 = cf_matrix/np.sum(cf_matrix, axis=1)
plt.figure(figsize=(8,6))
sns.heatmap(cf_matrix_n1, xticklabels=CATEGORIES, yticklabels=CATEGORIES, annot=True)
<Axes: >
Observation:
Classification Report for each class
Precision: precision is the fraction of relevant instances among the retrieved instances.
Recall: recall is the fraction of relevant instances that were retrieved.
F1 score: The F1 score is the harmonic mean of precision and recall, reaching its optimal value at 1 and its worst value at 0.
Model 1 ( Limited Layers )
from sklearn.metrics import classification_report
print(classification_report((normal_y_test), y_test_pred_classes_ln))
precision recall f1-score support
0 1.00 0.04 0.07 26
1 0.91 0.74 0.82 39
2 0.88 0.76 0.81 29
3 0.76 0.90 0.83 61
4 0.64 0.64 0.64 22
5 0.85 0.58 0.69 48
6 0.62 0.91 0.74 65
7 0.83 0.68 0.75 22
8 0.64 0.90 0.75 52
9 0.65 0.65 0.65 23
10 0.93 0.76 0.84 50
11 0.78 0.82 0.79 38
accuracy 0.75 475
macro avg 0.79 0.70 0.70 475
weighted avg 0.78 0.75 0.73 475
Model 2 ( Additional Layers)
from sklearn.metrics import classification_report
print(classification_report((normal_y_test), y_test_pred_classes_ln3))
precision recall f1-score support
0 0.40 0.38 0.39 26
1 0.83 0.90 0.86 39
2 0.68 0.93 0.78 29
3 0.95 0.85 0.90 61
4 0.71 0.77 0.74 22
5 0.82 0.67 0.74 48
6 0.78 0.66 0.72 65
7 0.78 0.82 0.80 22
8 0.74 0.88 0.81 52
9 0.78 0.61 0.68 23
10 0.86 0.88 0.87 50
11 0.76 0.82 0.78 38
accuracy 0.78 475
macro avg 0.76 0.76 0.76 475
weighted avg 0.78 0.78 0.77 475
Model 4 (With Transfer)
from sklearn.metrics import classification_report
print(classification_report((normal_y_test), y_test_pred_classes_ln4))
precision recall f1-score support
0 0.48 0.46 0.47 26
1 0.86 0.79 0.83 39
2 0.73 0.83 0.77 29
3 0.91 0.84 0.87 61
4 0.58 0.64 0.61 22
5 0.76 0.54 0.63 48
6 0.65 0.86 0.74 65
7 0.84 0.73 0.78 22
8 0.72 0.79 0.75 52
9 0.67 0.43 0.53 23
10 0.78 0.80 0.79 50
11 0.69 0.71 0.70 38
accuracy 0.73 475
macro avg 0.72 0.70 0.71 475
weighted avg 0.74 0.73 0.73 475
Efficiency improvement: AI and CNN can signficantly reduce the time required for plant seedling identification compared to manual labor. Invest in AI solutions to streamline the identification process, improving overall efficiency in plant management and monitoring
Labor Optimization: By automating the identitifacation of plan seedlings, manual labor can be redirected towards higher order decision making tasks. Implment AI technologies to augment human effort allowing skilled labor to focus on strategic planning, problem-solving and decision-making for good farm management.
Enhanced Crop management: AI driven identification can lead better crop management practices resuling in improved yiedls. Integrate AI technolies into crop management system to optimize planting, watering, and harvesting practices to enhance the overall agricultural productivity
*